Semi-markov Decision including an Unknown
نویسنده
چکیده
SEMI-MARKOV DECISION INCLUDING AN UNKNOWN Masami Kurano Chiba University PROCESSES PARAMETER (Received February 27, 1984: Revised May 8,1985) We consider the problem of minimizing the long-run average (expected) cost per unit time in a semiMarkov decision process including an unknown parameter. In the case of general state and action spaces and compact parameter space we construct the adaptive policy which has good properties under some identifiability conditions weaker than those for the strong consistency of the estimator. As example, we treat the age replacement with an unknown failure distribution.
منابع مشابه
A neural reinforcement learning model for tasks with unknown time delays
We present a biologically based neural model capable of performing reinforcement learning in complex tasks. The model is unique in its ability to solve tasks that require the agent to make a sequence of unrewarded actions in order to reach the goal, in an environment where there are unknown and variable time delays between actions, state transitions, and rewards. Specifically, this is the first...
متن کاملApplying Semi-Markov Models for forecasting the Triple Dimensions of Next Earthquake Occurrences: with Case Study in Iran Area
In this paper Semi-Markov models are used to forecast the triple dimensions of next earthquake occurrences. Each earthquake can be investigated in three dimensions including temporal, spatial and magnitude. Semi-Markov models can be used for earthquake forecasting in each arbitrary area and each area can be divided into several zones. In Semi-Markov models each zone can be considered as a sta...
متن کاملSystem-theoretical algorithmic solution to waiting times in semi-Markov queues
Markov renewal processes with matrix-exponential semi-Markov kernels provide a generic tool for modeling auto-correlated interarrival and service times in queueing systems. In this paper, we study the steady-state actual waiting time distribution in an infinite capacity single-server semi-Markov queuewith the auto-correlation in interarrival and service timesmodeled byMarkov renewal processes w...
متن کاملSolving Generalized Semi-Markov Processes using Continuous Phase-Type Distributions
We introduce the generalized semi-Markov decision process (GSMDP) as an extension of continuous-time MDPs and semi-Markov decision processes (SMDPs) for modeling stochastic decision processes with asynchronous events and actions. Using phase-type distributions and uniformization, we show how an arbitrary GSMDP can be approximated by a discrete-time MDP, which can then be solved using existing M...
متن کاملAvailability analysis of mechanical systems with condition-based maintenance using semi-Markov and evaluation of optimal condition monitoring interval
Maintenance helps to extend equipment life by improving its condition and avoiding catastrophic failures. Appropriate model or mechanism is, thus, needed to quantify system availability vis-a-vis a given maintenance strategy, which will assist in decision-making for optimal utilization of maintenance resources. This paper deals with semi-Markov process (SMP) modeling for steady state availabili...
متن کامل